Class-Based Weighted NB for Text Categorization

Naïve Bayes classifier is a supervised and probabilistic learning method (Manning, Raghavan, & Schuetze, 2008) which greatly simplifies learning by making the assumption that provided features are conditionally independent. Although this assumption usually does not hold, this classifier proves to compete well with other more sophisticated techniques (Rish, 2001). Moreover, being fast and easy to implement has resulted in frequent use of Naïve Bayes for text classification (Rennie, Shih, Teevan, & Karger, 2003). Studies comparing classification algorithms prove that Naïve Bayes is comparable in performance with decision trees and neural network classifiers (Han, & Kamber 2006). Many enhancements have been proposed so as to relax this unrealistic assumption. These enhancements are mainly in the area of feature selection and feature weighting (Lee, Gutierrez, & Dou, 2011). Feature selection is the process of selecting a subset of proposed features and using only these selected features in text categorization. Feature selection results in two main advantages: Firstly, by decreasing the amount of the effective vocabularies it makes classification more efficient. Secondly, it eliminates noise features and consequently makes classification more accurate (Manning et al., 2008). Feature weighting which obviously assigns a weight to each feature is more flexible than feature selection since feature weighting assigns continuous weights to features while feature selection assigns only 0/1 values (Lee et al., 2011). Many improvements have been proposed in both areas, but weight adjusting considering class attribute has rarely been investigated. In this chapter, we will propose the class-based weighted Naïve Bayes algorithm. In this algorithm, weight adjustment is performed for all samples with the same class attribute in the training dataset. Weight adjustment is achieved by examining different weights for each and every feature in the dataset and selecting the weight which contributes to the best improvement in the classification result. This mechanism will be elaborated further in section 3. This chapter is structured as follows. In the next section, we will provide a brief review of other enhancements proposed to improve Naïve Bayes classifier. In section 3, we will introduce our proposed algorithm, class-based weighted Naïve Bayes algorithm and show the results of our experiments. Finally, a direction for future research and conclusion are given.